Technical Field
This application relates to driver assistance systems and methods to detect the vertical deviation of a contour of a road using a camera, and more specifically to using a stabilized coordinate frame to detect vertical deviation of a contour of the road.
Description of Related Art
In recent years, camera-based driver assistance systems (DAS) have been entering the market, including lane departure warning (LDW), automatic high-beam control (AHC), traffic sign recognition (TSR), forward collision warning (FCW) and pedestrian detection.
Reference is now made to FIGS. 1 and 2 which illustrate a system 16, including a camera or image sensor 12 mounted in a vehicle 18, according to some embodiments. Image sensor 12, imaging a field of view in the forward direction, provides image frames 15 in real time and image frames 15 are captured by an image processor 30. Processor 30 may be used to process image frames 15 simultaneously and/or in parallel to serve a number of DAS/applications. Processor 30 may be used to process image frames 15 to detect and recognize an image or portions of the image in the forward field of view of camera 12. The DAS may be implemented using specific hardware circuitry (not shown) with on-board software and/or software control algorithms in storage 13. Image sensor 12 may be monochrome or black-white, i.e., without color separation, or image sensor 12 may be color sensitive. By way of example in FIG. 2, image frames 15 are used to serve pedestrian detection 20, TSR 21, FCW 22 and real time detection 23 of the vertical contour of the road or deviation from the road plane according some embodiments.
In some embodiments, more than one camera may be mounted in a vehicle. For example, a system may have multiple cameras pointing in different directions. A system also may have multiple cameras pointing in the same or similar directions with respect to the vehicle, but mounted at different locations. In some embodiments, a system may have multiple cameras that have partially or completely overlapping fields of view. In some embodiments, two side by side cameras may operate in stereo. The non-limiting examples discussed herein contemplate single-camera systems, but they may be similarly implemented in multiple-camera systems, wherein some or all of the relevant images and frames may be captured by different cameras, or may be created from a composite of images captured from multiple cameras.
In some cases, image frames 15 are partitioned between different driver assistance applications and in other cases the image frames 15 may be shared between the different driver assistance applications.
Certain existing methods for detecting the vertical deviation of a contour of a road using a vehicle-mounted camera are known. Certain previously known algorithms can be summarized as follows:                1. A first pair of two consecutive frames captured by a vehicle-mounted camera are aligned using a homography of the road. This gives the ego-motion (rotation R and translation T) between the frames. FIG. 3A shows a grid of 33 points in the image which are tracked and used to compute the homography.        2. A second pair of frames is then selected, the current frame and the nearest previous frame representing a point in time from which the vehicle has currently moved over a minimum threshold distance. A chaining of the first pair of frames (the consecutive frames) is used to create an initial guess of the homography, and a more accurate homography for the second pair of frames is then computed, and the reference plane is determined.        3. The path of the vehicle is then projected onto the image plane (as shown by the lines in FIG. 3C). Strips along each path are used to compute the residual motion, which gives the profile relative to the reference plane determined. FIG. 3B shows a normalized correlation score for a strip 31 pixels wide along the path of the left wheel for vertical motion ±6 pixels. A small curvature can be seen, indicated by the arrow. This curvature represents the small residual motion indicative of a speed bump.        4. Finally, the residual motion is translated into a metric distance and height, and is combined into a multi-frame model. The results are shown in the upper graphs in FIG. 3C.        
One previously known algorithm in accordance with the brief description above is described in greater detail in U.S. Pat. No. 9,256,791.
One example of data generated in accordance with previous methods is shown in FIGS. 4A and 4B. FIGS. 4A and 4B show road profiles from a sequence of frames aligned using the recovered rotation (R), translation (T) and plane normal (N). One can see a speed bump near meter 40, but also a strong divergence of signals caused by inaccurate motion and plane normal; the samples show a divergence of about 0.1 meters in height between the profiles.